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REMARKS 

Claims 1-26 are pending in this application. By this Amendment, the specifciation, 
claims 1-4, 7, 9, 11 and 13 are amended and claims 14-26 are added. Reconsideration in view 
of the above amendments and the following remarks is respectfully requested. 

Applicants gratefully acknowledge the Office Action's indication that claims 7, 9 and 

11- 13 define patentable subject matter. However, for at least the reasons set forth below, 
Applicants respectfully submit all pending claims are in condition for allowance. 

A. The Office Action rejects claims 1, 3-4 and 8 under 35 U.S.C. §103(a) over U.S. 
Patent No. 5,835,667 to Wactlar et al. (hereafter "Wacdar") and U.S. Patent No. 6,411,724 to 
Vaithilingam et al. (hereafter cf Vaithilingam"). The rejection is respectfully traversed. 

The Office Action asserts that Wacdar discloses (b) detecting change of retrieval 
environment based on the retrieval performance evaluation citing column 11, lines 66-column 
12 line 5 of Wacdar and (c) updating ... the reliability of the weight by reflecting the retrieval 
performance evaluation and the retrieval environment change (emphasis added) citing column 
12 lines 47-51 of Wacdar. 

Wacdar discloses a method and apparatus of creating a digital library. See lines 1-14 of 
the Abstract Applicants respectfully submit that in creating a digital library (i.e., column 7, line 

12- column 14, line 37), Wacdar discloses paragraphing function 33 (i.e., column 11, line 54- 
column 13, line 51) used to segment a video data into "paragraphs" that can reasonably be 
abstracted by a "representative frame' 5 and treated as a unit for an image content search. See 
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column 11, lines 66-67. Icons can be created by function 35 for subsequent presentation to a 
user performing a search. See column 13, lines 52-55. 

Paragraphing function 33 in Wactlar determines paragraph boundaries by identifying 
beginning and end points for each shot ... by interpreting image sequences in the video data. See 
column 11, lines 58-61. In this paragraph boundary context, Applicants respectfully submit 
Wacdar discloses detection of big "image changes." See column 12, lines 1-5. Further, 
previously identified information of the video data 20 can be used to increase the reliability of 
the beginning and end point of the paragraph boundaries. See Figure 6 and column 12, lines 41- 
51. 

Thus, Applicants respectfully submit that the paragraphing function 33 in Wactlar 
asserted in the Office Action is directed to segmenting video data 20 to create a searchable item 
for a digital library or database 36. In contrast, claim 1 is directed to a method for updating 
multimedia feature information in a multimedia retrieval system using weight of multimedia 
features and reliability of the weight and recites evaluating a retrieval performance of a search 
conducted on searchable items. Accordingly, Applicants respectfully submit that the 
paragraphing function 33 in Wactlar defines end points of segments of video data 20 and does 
not teach or suggest at least a feature a method for multimedia retrieval based on a query and 
therefore cannot teach or suggest a method for (c) updating the weight ... and reliability of the 
weight by reflecting the retrieval performance evaluation. 
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In contrast to the assertion in the Office Action, Applicants respectfully submit that 
column 12, lines 47-51 and Figure 6 in Wactlar disclose improved reliability for determining 
beginning and end points in "paragraphs" described above using the paragraph function 33. 
Thus, Wactlar discloses improving the reliability of a feature being segmented (that could later 
possibly be searched for) from a video data 20 to be entered into a digital library. 

Thus, Applicants respectfully submit that Wactlar does not teach or suggest updating 
the.. .reliability of the weight by reflecting the retrieval performance evaluation and combinations 
thereof as recited in claim 1. Again, claim 1 recites updating multimedia feature information of 
a retrieval performance of a multimedia retrieval system. 

Applicants respectfully submit that Wactlar does disclose querying a digital library. See 
a Section entitled "Exploration of the Digital Library" in column 14, line 39-column 17, line 26. 
However, column 14 lines 52-61 in Wactlar merely discloses a general query function. Thus, 
Applicants respectfully submit that Wactlar does not teach or suggest at least the specific features 
and combinations thereof recited in claim 1. 

Finally, the Office Action admits Wacdar does not teach or suggest updating ... 
reliability of the weight by reflecting the retrieval performance evaluation and the retrieval 
environment change (emphases added). Applicants respectfully submit that Vaithilingam does 
not teach or suggest features recited in claim 1 and lacking from Wacdar. Therefore, Wactlar 
and Vaithilingam, individually or in combination do not teach or suggest at least features of (b) 
detecting change of retrieval environment and (c) updating the weight ... and reliability of the 
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weight by reflecting the retrieval performance evaluation and the retrieval environment change 
and combinations thereof as recited in claim 1. Further as discussed below, Vaithilingam does 
not teach or suggest various features of the present invention as recited in claims 1-13. 

For at least the reasons set forth above, Applicants respectfully submit claim 1 defines 
patentable subject matter. Claims 3-4 and 8 depend from claim 1 and therefore define 
patentable subject matter for at least that reason as well as their additionally recited features. 
Withdrawal of the rejection of claims 1, 3-4 and 8 under 35 U.S.C. §103 is respectfully requested. 

B. The Office Action rejects claims 2 and 10 under 35 U.S.C. §103(a) over 
Vaithilingam and U.S. Patent No. 6,408,293 to Aggarwal et al. (hereafter "Aggarwal"). The 
rejection is respectfully traversed. 

Applicants respectfully submit that Vaithilingam discloses defining meta-descriptors, 
which are information about descriptors, and the descriptors are representations of a feature or 
distinctive characteristic of a multimedia object. See the Abstract and column 12, lines 12-32. 
Vaithilingam discloses querying a multimedia repository by formulating a query (131), acquiring 
descriptor/s and meta-descriptor/s for a repository multimedia item (132), comparing the 
descriptor/ s of the repository multimedia item and the query multimedia item (135), and ranking 
and displaying the results (136/137). See the Abstract, column 9, lines 1-28 and Fig. 2 of 
Vaithilingam. 
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Further, a method of optimizing a meta-descriptor performs a multimedia query 
process then ranks and display results. If the results are not OK, a user can modify the meta- 
descriptor. See Fig 3 and col. 9, lines 29 to 65 of Vaithilingam. 

With respect to claim 2, the Office Action asserts that Vaithilingam discloses (c) 
calculating retrieval performance with respect to the results of present retrieval citing column 
5, lines 40-54 of Vaithilingam, where "performance" is read on "how clearly defined and 
compact the clusters are." 

As described in Vaithilingam, mathematically, a cluster is an aggregation of points in the 
test space such that the distance between any two points in the cluster is less than the distance 
between any point on the cluster and any point not in it. Vaithilingam discloses clustering (112) 
multimedia data in a database based on the descriptors as a way of grouping similar multimedia 
information . See column 5, lines 22-25. Vaithilingam discloses that metadescriptors can then 
be assigned (113) to each cluster. The next step (114) is to attach metadescriptors to multimedia 
information based on cluster information. See column 6, lines 30-32 and column 7, lines 38-40. 
Vaithilingam discloses meta-descriptors being binary vectors, weighted vectors or a string 
notation such as a character string. See column 6, lines 47-67 and claims 1-6 of Vaithilingam. 
Thus, "compactness" of a cluster or the like is merely a description of the cluster, which can be 
used when extracting or assigning a corresponding metadescriptor and is not related to a querying 
function . Applicants respectfully submit that metadescriptors as described in Vaithilingam are 
a secondary grouping in contrast to a listing of a collection of multimedia items. 
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Applicants respectfully submit that Vaithilingam discloses a method for retrieving 
multimedia information. See column 9, lines 1-65 and Figs. 2 and 4. Further, Vaithilingam 
discloses metadescriptors being a scheme for accessing descriptors of multimedia data in a 
database. Querying a multimedia database can use the metadescriptors by searching all clusters 
in a database then using a comparison (135) between the query descriptor and the descriptor for 
the repository item. See column 9, lines 14-28. Accordingly, Applicants respectfully submit that 
metadescriptors represent information about a group or cluster and can be used to group 
multimedia data to access descriptors via clusters in performing a query on a database. 

As such, Applicants respectfully submit that metadescriptors are directed to a cluster of 
multimedia items and indirecdy provide access to descriptors. Further, metadescriptors merely 
disclose information about characteristics of a cluster and are not directed to modifying individual 
features of a multimedia a item such as a weight. Accordingly, Applicants respectfully submit that 
metadescriptors do not teach or suggest modifying a "weight" of a multimedia item through using 
reliability, let alone updating the reliability of the present weight by reflecting the calculated 
retrieval performance and updating the present weight using the updated reliability and 
combinations thereof as recited in claim 2. 

Further, the Office Action asserts Vaithilingam discloses updating the reliability of the 
present weight by reflecting the calculated retrieval performance as recited in claim 2 citing 
column 9, lines 56-65. Applicants respectfully submit that a method for querying a multimedia 
repository is illustrated by principal steps 131-137 shown in Figure 1 and described in column 9, 
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lines 1-28. Column 9, lines 56-65 disclose a technique for optimizing meta-descriptors that 
formalizes user input as illustrated in Figure 4. As such, lines 52-54 of column 9 disclose the user 
can determine the weights for each of the suitable features. Thus, Applicants respectfully submit 
Vaithilingam discloses updating the weights of individually selected multimedia items based on 
user input to determine present weights. This could be considered similar to updating a present 
weight based on using one or more user feedbacks as recited in claim 2. Upon completion of the 
user updating individual feature weights, Vaithilingam discloses the retrieval system then updating 
the metadescriptors (306), which were defined above (301) (e.g., 112-114) for example bv 
assigning the new weights (e.g., user input). See column 9, lines 56-60. Thus, Applicants 
respectfully submit the metadescriptors are updated based on the weights and therefore does not 
teach or suggest at least a feature of updating the present weight using the updated reliability and 
combinations thereof as recited in claim 1. Thus, Applicants respectfully submit that 
metadescriptors do not teach or suggest reliability of a present weight that reflects calculated 
retrieval performance or retrieval performance. 

For the reasons set forth above, metadescriptors and querying in Vaithilingam do not 
teach or suggest at least a features of calculating retrieval performance with respect to the results 
of present retrieval using the one or more user feedbacks, updating the reliability of the present 
weight by reflecting the calculated retrieval performance and updating the present weight using 
the updated reliability and combinations thereof as recited in claim 2. Applicants respectfully 
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submit that Aggarwal does not teach or suggest features recited in claim 2 and lacking from 
Vaithilingam. 

Aggarwal discloses information fed-back by the user during intra-query modification is 
used for intra-object learning of the user's perception to expedite inter-object learning of a user's 
perception. See the Abstract of Aggarwal. As shown in Figure 3, an intra-query object 
processing block 1000 includes an intra-query object learning function 301 that receives user 
feedback 302. As shown in Figure 4, function 301 segments an input multimedia object to 
receive user input with respect to the segmentation grouping until the user indicates their 
satisfaction with the arranged object. The retrieved objects are used to perform database retrieval 
in function block 303. If the user is satisfied then the query stops, otherwise the process enters 
the prior art inter-query object processing block 2000. See Figures 3-4 and column 3, line 29- 
column 4, line 48 of Aggarwal. Aggarwal further discloses a similarity measure that is an adaptive 
quality adjusted using user feedback. See column 5, lines 3-15. 

Accordingly, Applicants respectfully submit that user feedback are used to update a 
feature (e.g., define or redefine the retrieved object or indicate relative importance of features) 
based on user feedback, but does not teach or suggest updating a reliability of the retrieval 
performance . Thus, Applicants respectfully submit that Aggarwal does not teach or suggest at 
least a feature of calculating retrieval performance using the one or more user feedbacks and 
combinations thereof as recited in claim 2. 
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For at least the reasons set forth above, Applicants respectfully submit claim 2 defines 
patentable subject matter. For at least reasons similar to claim 2, Applicants respectfully submit 
claim 10 defines patentable subject matter. Withdrawal of the rejection of claims 2 and 10 under 
35 U.S.C. §103 is respectfully requested. 

C. The Office Action rejects claims 5 and 6 under 35 U.S.C. §1 03(a) over Wactlar, 
Vaithilingam and Aggarwal. The rejection is respectfully traversed. 

As described above, claim 1 defines patentable subject matter over Wacdar and 
Vaithilingam. Applicants respectfully submit that Aggarwal does not teach or suggest at least 
features of detecting and updating and combinations thereof as recited in independent claim 1. 
Thus, Applicants respectfully submit that Wacdar, Vaithilingam and Aggarwal, individually or in 
combination, do not teach or suggest at least features of detecting and updating and combinations 
thereof as recited in independent claim 1. 

As described above, Applicants respectfully submit claim 1 defines patentable subject 
matter. Claims 5-6 depend from claim 1 and therefore also define patentable subject matter for 
at least that reason as well as their additionally recited features. Withdrawal of the rejection of 
claims 5-6 under 35 U.S.C. §103 is respectfully requested. 

D. Claims 14-26 are newly added by this Amendment and believed to be in 
condition for allowance. 
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CONCLUSION 



In view of the foregoing amendments and remarks, it is respectfully submitted that the 
application is in condition for allowance. Favorable consideration and prompt allowance are 
earnestly solicited. 

If the Examiner believes that any additional changes would place the application in better 
condition for allowance, the Examiner is invited to contact the undersigned attorney, Carl R. 
Wesolowski, at the telephone number listed below. 

To the extent necessary, a petition for an extension of time under 37 C.F.R. 1.136 is 
hereby made. Please charge any shortage in fees due in connection with the filing of this, 
concurrent and future replies, including extension of time fees, to Deposit Account 16-0607 and 
please credit any excess fees to such deposit account 



Respectfully submitted, 
FLESHNER & KIM, LLP 




Daniel Y.J.Kim 
Registration No. 36,186 
Carl R. Wesolowski 
Registration No. 40,372 



P.O. Box 221200 
ChantiUy,VA 20153-1200 
703 502-9440 DYK/cR\v:jid 
Date: March 4, 2003 



RECEIVED 

MAR 1 0 2003 
Technology Center 21 00 
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A. Specification Paragraphs With Mark-ups to Show Changes Made 
The following are mark-ups to show changes made to paragraph starting at page 
1, line 5 and ending at page 1, line 8: 

The present invention relates to a multimedia retrieval system, and more particularly to 
a method for updating multimedia feature information such as weights and reliability adaptively 
to reflect changes of multimedia retrieval environment and a data structure therefor. 

The following are mark-ups to show changes made to paragraph starting at page 
3, line 6 and ending at page 3, line 9: 

It is an object of the present invention to provide a method for updating reliability of 
multimedia features on the basis of a retrieval performance calculated by user feedback [to] 
considering a relevance of the previous multimedia retrieval. 



The following are mark-ups to show changes made to paragraph starting at page 
6, line 6 and ending at page 6, line 14: 

In the multimedia retrieval and multimedia feature weight update system using the 
multimedia feature weights and the reliability of the multimedia feature weights, the multimedia 
feature information structure is characterized in that a present multimedia retrieval is performed 
with previously stored multimedia feature weights and reliability thereof, and then a retrieval 
performance is calculated on the basis of user's feedback on the result of the present retrieval, 
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consequently, the reliability of the multimedia feature weights is updated in consideration [with] 
of present retrieval performance, and finally the present multimedia feature weights are updated 
on the basis of the updated reliability. 



The following are mark-ups to show changes made to paragraph starting at page 
6, line 15 and ending at page 6, line 18: 



In the present invention, the reliability is high if a value of the calculated retrieval 
performance is higher than that of the previous retrieval performance, and the reliability is low 
if the value of the calculated retrieval performance is lower than that of the previous retrieval 
performance. 



The following are mark-ups to show changes made to paragraph starting at page 
6, line 22 and ending at page 6, line 24: 

[There] The more the number of feedbacks participated in calculating the present 
retrieval performance, the higher the retrieval performance influences to the reliability update. 

" — ~ fTVi ■ 

The following are mark-ups to show changes made to paragraph starting at page 
8, line 15 and ending at page 8, line 21: 

Accordingly, in case when a retrieval environment such as the similarity [measurement 
method] measure is changed, the feature weights are quickly learned using the new feedback in 
the new retrieval environment by lowering the reliability of the previous weights. In case when 
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the retrieval environment is maintained without change such that the weights are learn in long 
time and stable, the feature weights are not quickly changed by heightening the reliability of the 
previous weights. 

The following are mark-ups to show changes made to paragraph starting at page 
11, line 6 and ending at page 11, line 11: 

When using the weight update method with the similarity, if the retrieval environment 
such as the similarity [measurement] measure is changed, the optimal weight is quickly obtained 
using new feedbacks so as to be adapted to the new environment by lowering the reliability of 
the previous weight, and if the retrieval environment are maintained without change such that 
the weight is stable, the weight is not easily changed by the new feedbacks. 
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Amended Claims With Mark-ups to Show Changes Made 

1. (Amended) A [Method] method for updating multimedia feature information in 
a multimedia retrieval system using weight of multimedia features and reliability of the weight, 
comprising [the steps of): 

(a) evaluating a retrieval performance using multimedia feature information; 

(b) detecting change of retrieval environment based on the retrieval performance 
evaluation; and 

(c) updating the weight of the multimedia feature information and reliability of 
the weight by reflecting the retrieval performance evaluation and the retrieval environment 
change. 

2. (Amended) A [Method] method for updating multimedia feature information in 
a multimedia retrieval system using weight of multimedia features and reliability of the weight, 
comprising [the steps of]: 

[(a)] retrieving multimedia using previous weight; 

[(b)] receiving one or more user feedbacks with respect to results of the 
multimedia retrieval; 

[(c)] calculating retrieval performance with respect to the results of present 
retrieval using the one or more user feedbacks; 

updating a present weight using the one or more user feedbacks; 
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[(d)] updating the reliability of the present weight by reflecting the calculated 
retrieval performance; and 

[(e)] updating the present weight using the updated reliability. 

3. (Amended) The method as claimed in claim 1, wherein [the] updating reliability 
[update] of the weight is proportionally influenced by the retrieval performance. 

4. (Amended) The method as claimed in claim 1, wherein [the] updating reliability 
[update] of the weight is proportionally influenced by improvement of the retrieval 
performances. 

7. (Amended) The method as claimed in claim [6] 1, wherein the reliability is 
calculated by a following formula: 

previous reliability x (1 + reliability increment) + a 
wherein, 

reliability increment: a function that multiplies the difference between the present 
and previous retrieval performance with the number of feedbacks. 

a: constant for making the reliability value proportional to the number of 
feedbacks in same condition. 
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9. (Amended) The method as claimed in claim [8] 1, wherein the reliability is 
calculated by a following formula: 

previous reliability x (1 + reliability increment) + a 
wherein, 

reliability increment: a function that multiplies the rate of the present retrieval 
performance to the previous retrieval performance with the number of feedbacks. 

a: constant for making the reliability value proportional to the number of 
feedbacks in same condition. 



1 1 . (Amended) A method for updating weight of multimedia features using reliability 
of the weight in a multimedia retrieval system using weight among multimedia features and 
weight among elements of the multimedia feature, wherein the weight is updated based on the 
following way that: 

[(a) the more times the previous feature weights are learned with the feedbacks 
from the user, the less the feature weights are influenced by new feedback; 

(b) the more recent the feedback is, the more the feedback influence to the feature 
weights update; and 

(c) ] {a) learning rate of the weights among the multimedia features is higher than 
that of the weights among elements of a multimedia feature. 
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13. (Amended) A multimedia data structure for retrieval of multimedia objects using 
weight among multimedia features and weight among elements of the multimedia feature, 
wherein the weight is updated based on the following way that: 

[(a) the more times the previous feature weights are learned with the feedbacks 
from the user, the less the feature weights are influenced by new feedback; 

(b) the more recent the feedback is, the more the feedback influence to the feature 
weights update; and 

(c) ] £a} learning rate of the weights among the multimedia features is higher than 
that of the weights among elements of a multimedia feature[, in relation to the reliability formula, 
[Reliability 2 x 01d_W + Cur_W] / [Reliability 3 + 1] wherein, 0 < a < 1, and exponential term 
"a" in the formula for weights of features is less than exponential term "a" in the formula for 
weights of elements of a feature]. 
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